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Predict  System  Failures 
Before  You  Build 


Question:  “I  have  a  great  idea,  but 
what  happens  if  I  change  my 
system  architecture?” 


►  Eagle6  is  a  modeling  and 
simulation  tool  that  is  capable 
of  predicting  system  behavior. 


/ 

Ensure  Changes  to  System 

Architecture  Do  Not  Cause 

1 — 

Unintended  System  Behavior 

| 

1 

i 

/ 

Validates  Architectural  Designs 

That  Mix  Legacy  Systems  and  New 

L 

Technologies 

i 

[ 


Evaluates  Business  Process 
Reengineering  Designs  (Lean  Six 
Sigma) 


Answer:  Test  BEFORE  You  Invest! 
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System  of  System  Dependencies 


>lem:  What  could  happen  if  I  upgrade/change  this  system? 


Coming  Up:  How  we  solve  this  problem... 


Problem  Statement 


“System  Architecture  Designs  May  Contain 
Unintended  and/or  Unknown  System  Behavior.” 


Issues  with  Requirements 

Typical  difficulties 

>  Users  initial  concept  of  system  is  nebulous 

>  Users  description  of  system  is  incomplete  and  inconsistent 

>  Users  (usually)  don't  understand  what  they  really  need 

>  Interpreting  users  description  of  problem  is  error-prone 

>  Perception  of  system  changes  during  analysis,  requires  reworking 

>  Different  users  will  view  the  system  differently 


Question:  How  do  we  know  if  the  system  architecture  represents  all  system 
requirements? 

Answer:  Model  Checking 


Produce  Better  Software 


►  Major  goal  of  software  engineers 

°  Develop  reliable  systems 

►  Formal  Methods 

°  Mathematical  languages,  techniques  and  tools 
°  Used  to  specify  and  verify  systems 

°  Goal:  Help  engineers  construct  more  reliable 
systems 

►  A  mean  to  examine  the  entire  state  space  of 
design  (whether  hardware  or  software) 

°  Establish  a  correctness  or  safety  property  that  is 
true  for  all  possible  inputs 


Problems  with  Formal  Methods 


► 


Past  years  of  the  formal  methods 


°  Obscure  notation 
°  Non-scalable  techniques 
°  Inadequate  tool  support 
°  Hard  to  use  tools 
°  Very  few  case  studies 
°  Not  convincing  for  practitioners 


►  Bottom  Line:  It’s  not  easy. 


h 


Unknown  System  Behavior 


Model  Checking 

Pros 

►  Great  for  system  safety 
testing. 

°  Medical  Systems 
°  Weapon  Systems 

►  Great  for  finding 
unknown  system 
behavior  and/or 
architectural  design 
flaws  (assertion 
checking) 


gf^EAGLE6 

lJCSmooeling 


Cons 

Modeling  languages 
are  very  complex  and 
require  domain 
expertise 

Models  require  a  very 
long  time  to  develop 

Modifying  models  is 
not  easy,  making  reuse 
very  difficult 


Test  Using  One  Model 


Requirements  Testing 


Conformance  Testing 


Functional  Testing 


Integration  Testing 


Black  Box  Testing 


Performance  Testing 


Regression  Testing 


System  Testing 


Using  a  single 
model  allows  for 
system  constraints 
|  to  remain  resident 
throughout  all 
I  stages  of  the 
k  system  lifecycle. 


Unit  Testing 


Summary  of  Formal  Methods 


►  Formal  methods  can  be  applied  at  various 
points  through  the  development  process 

°  Specification 
°  Verification 

►  Specification:  Give  a  description  of  the  system 
to  be  developed,  and  its  properties 

►  Verification:  Prove  or  disprove  the  correctness 
of  a  system  with  respect  to  the  formal 
specification  or  property 
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Eagle6  Demonstration 
(www.Eagle6.com) 


Old  SoS  Problem,  New  Solution 


Problem:  What  could  happen  if  I  upgrade/change  this  system? 


Answer:  Model  the  solution  and  test,  test,  test! 


Reminder:  Test  Using  One  Model 


Requirements  Testing 


Conformance  Testing 


Functional  Testing 


Integration  Testing 
Black  Box  Testing 


Performance  Testing 


Regression  Testing 


System  Testing 


Using  a  single 
model  allows  for 
system  constraints 
|  to  remain  resident 
throughout  all 
I  stages  of  the 
k  system  lifecycle. 
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Model  the  Enterprise  Without  a 
Single  Line  of  Code 


EagleG  Model  in  Text  Form 


Eagle6  UI  -  Windows  Internet  Explorer 


http://db001d:8088/main.html?modeljd=16 


-  A 


i|l  Google 


File  Edit  View  Favorites  Tools  Help 


Google 


▼  il  Search-  More» 


Favorites  Suggested  Sites  ▼  Get  More  Add-ons  ▼ 


:  :  ”  =  US  Airways  |  Airl...  Eagle6  UI 


i  Military  Symbols  M  US  Airways  |  Airl... 


a  -  □  - 


IEAGLE6 

Imodeling 


Preview  Code 


x  I 


; 


Assertions 
Scenario  Viewer 
Naval  Gunship 

R2D_activity 
CD_activity 
GCC_activity 
]  R3D_activity 
GCC2_activity 
GMP_activity 
\  GMCP_activity 
!••  GMP2_activity 
'  CDC_acrivity 
!  EOD_activrty 
GM_activity 


Scenario  Generation  COMPLETED  View 


ROOT  R2D_activity:  {*  <1>  R2D_displayNewTarget  *}; 

ROOT  CD_activity:  {*  <1>  CD_spotNewTarget  *}; 

ROOT  GCC_activity:  {*  <0-1/0. 2,0. 8>  GCC_setTarget  *}; 

ROOT  R3D_activity:  {*  <0-l/0.28,0.72>  R3D_setTarget  *}; 

ROOT  GCC2_activity :  {*  <0-l/0.44128,0.55872>  GCC_openFire  *}; 

ROOT  GMP_activity:  {*  <0- 1/0.452454, 0.547546>  GMP_answerRequest_GCC_openFire  *}; 

ROOT  GMCP_activity:  {*  <0-1/0.45793, 0.54207>  GMCP_answerFireRequest  *}; 

ROOT  GMP2_activity:  {*  <0-1/0.463351, 0.536649>  GMP_answerRequest_GMCP_ossData  *}; 

ROOT  CDC_activity:  {*  <0-1/0. 490183, 0.509817>  CDC_answerRequest_GMP_ossData  *}; 

ROOT  EOD_activity :  {*  <0- 1/0.495281, 0.504719>  EOD_answerRequest_CDC_ossData  *}; 

ROOT  GM_activity:  {*  <0-1/0. 505013, 0.494987>  GM_answer_GMCP_openFireCommand  *>; 

El:  (  GCC_targetNotSet  CD_targetLost  ); 

Bl:  (  CD_request_GCC_openFire  CD_wait_GCC_openFire  (  GCC_openFireFailed  |<0.25249031177832>  targetMissed  |<0.58914406084842>  targetHit )  ); 
Dl:  (  GCC_targetSet  CD_followTarget  (  CD_abortTarget  |<0.8>  Bl  )  ); 

Gl:  (  CD_request_GCC_setTarget  CD_wait_GCC_setTarget  (  El  |<0.873>  Dl  )  ); 

CD_spotNewTarget:  (  R2D_displayNewTarget  (  CDJgnoreTarget  |<0.8>  Gl  )  ); 

B2:  (  R3D_targetNotSet  GCC_targetNotSet ); 

A2:  (  R3D_targetSet  GCC_ta  rgetSet ); 

D2:  (  GCC_request_R3D_setTarget  GCC_wait_R3D_setTarget  (  B2  |<0.97>  A2  )  ); 

GCC_setTarget:  (  CD_request_GCC_setTarget  (  GCC_targetNotSet  |<0.9>  D2  )  ); 

R3D_setTarget:  (  GCC_request_R3D_setTarget  (  R3D_targetNotSet  |<0.97>  R3D_targetSet )  ); 

A3:  (  GMP_openFireFailed  GCC_openRreFailed  ); 

B3:  (  GCC_request_GMP_openFire  GCC_wait_GMP_openFire  (  A3  | <0.257643175284>  targetMissed  |  targetHit )  ); 

GCC_openFire:  (  CD_request_GCC_o  pen  Fire  (  GCC_o  pen  Fire  Failed  |<0.98>  B3  )  ); 

A4:  (  GMCP_openFireFailed  GMP_o  pen  Fire  Failed  ); 

B4:  (  GMCP_displayOpenFireRequest  (  A4  |<0.6072398071>  targetHit  | <0.2602456316>  targetMissed  )  ); 

GMP_answerRequest_GCC_openFire:  (  GCC_request_GMP_openFire  (  GMP_openFireFailed  |<0.99>  E14  )  ); 

D5:  (  GMCP_failReceiving_GMP_ossData  GMCP_o  pen  Fire  Failed  ); 

A5:  (  GM_openFireFailed  GMCP_openRreFailed  ); 

B5:  (  GMCP_send_GM_openRreCommand  GMCP_wait_GM_openFireCommand  (  A5  |<0.665>  targetHit  |<0.285>  targetMissed  )  ); 

C5:  (  GMCP_receive_GMP_ossData  B5  ); 

E5:  (  GMCP_request_GMP_ossData  GMCP_wait_GMP_ossData  (  D5  | <0.9223662294>  C5  )  ); 

GMCP_answerFireRequest:  (  GMCP_displayOpenRreRequest  (  GMCP_openRreFailed  |<0.99>  E5  )  ); 

B6:  (  GMP_failReceiving_CDC_ossData  GMCP_failReceiving_GMP_ossData  ); 

A6:  (  GMP_receive_CDC_ossData  GMCP_receive_GMP_ossData  ); 

C6:  (  G M P_requ es t_CDC_o ssDa ta  GMP_wait_CDC_ossData  (  B6  |<0.95089302>  A6  )  ); 

GMP_answerRequest_GMCP_ossData:  (  GMCP_request_GMP_ossData  (  GMCP_failReceiving_GMP_ossData  |<0.95>  C6  )  ); 

B7:  (  CDC_failReceiving_EOD_ossData  GMP_failReceiving_CDC_ossData  ); 

A7 :  (  CDC_receive_EOD_ossData  GMP_receive_CDC_ossData  ); 

C7 :  (  CDC_request_EOD_ossData  CDC_wait_EOD_ossData  (  B7  |<0.960498>  A7  )  ); 


Sji  Local  intranet  |  Protected  Mode:  Off 
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Eagle6  Graphical  Editor 


=!  US  Airways  |  Airl...  Eagle6  UI 


MODELING 


;  Assertions 
f  Scenario  Viewer 
J  Naval  Gunship 


R2D_activity 

R2D_displayNewTarget 

CD_activity 

CD_spotNewTarget 

r  R2D_displayNewTarget 

i 

CD_ignoreTarget 

!'-  G1 

GCC_activity 

R3D_activity 

GCC2_activity 

GMP_activity 

GMCP_activity 

GMP2_activity 

CDC_activity 

EOD_activity 

GM_activity 


Test  SoS  Assertions 


Requirements  Testing 

Conformance  Testing 

Functional  Testing 


Integration  Testing 


Black  Box  Testing 


Performance  Testing 


Regression  Testing 
System  Testing 

Unit  Testing 


|oSr  g  US  Airways  |  Airl...  M  Eagle6  UI  3  Military  Symbols 
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;  Assertions 
f-  Scenario  Viewer 
*  Naval  Gunship 

#"  R2D_activity 

!-  R2D_displayNewTarget 
CD_activity 

CD_spotNewTarget 

R2D_displayNewT 
CDjgnoreTarget 
G1 

GCC_activity 
R3D_activity 
GCC2_activity 
GMP_activity 
GMCP_activity 
GMP2_activity 
CDC_activity 
EOD_activity 
■  G  Inactivity 
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Naval  Gunship  Test  Scenarios 

0  Test  Category:  SHOW  ALL 


Test  Type:  Show  All 


SHOW  ALL 


Category: 

Test 


Event 


□ 

□ 

□ 

□ 

□ 

□ 


GMP_openFireFailed 

GMjaunchMissile 

GMP_openFireFailed 

AS 

targetMissed 

GCC_targetSet 


Occurs  N  times 

>  90 

>=  i 

=  o 

<  4 

>  2 

>  0 


Min 

Scope 

1 


Event  ACCEPTANCE  TESTING 
BLACK  BOX  TESTING 
COMPATIBILITY  TESTING 
CONFORMANCE TES^NG 
FUNCTIONAL  TESTING 
INTEGRATION  TESTING 
LOAD  TESTING 
PERFORMANCE TESTNG 
REGRESSION  TESTING 
SMOKE  TESTING 
STRESS  TESTING 
SYSTEM  TESTING 
UNIT  TESTING 
WHITE  BOX  TESTING 


Test  All  I  Add  New  Test 


Sequenced  Events 


Category: 

Test  Attribute 

Max  Watts 


Min 

Sum  Result 

Scope 


Failed  Scenarios 


ox 

ox 

ox 

OX 

ox 

OX 


ox 
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Key:  Tests  via  Queries 


1)  Event  Count  -  Check  for  the  existence  of  a 
specific  system  state 

2)  Sequence  of  Events  -  can  a  series  of  events 
happen? 

3) Simultaneous  Events  -  Can  a  combination  of 
events  happen? 
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Run  All  System  Tests  with  One 
Model  in  Seconds 


Scenario  Viewer 


Eagle6  Summary 


1 .  Modeling  and  Simulation  software  tool  that  is  used  to  dynamically  model  any 
type  of  complex  enterprise  system  to  identify  risks  in  system  architecture. 

2.  Checks  all  possible  system  states  within  the  model  scope. 

3.  Capable  of  executing  all  types  of  system  tests  within  a  single  model. 

4.  Modeling  interface  that  allows  a  user  to  write  models  without  having  to  learn  a 
modeling  language  (tool  is  designed  for  the  average  user) 
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RIVERA 

GROUP 


Questions? 

Dr.  Joey  Rivera 
irivera@riverainc.com 


Phil  Lushin 

plushin@riverainc.com 

812-246-4055 


